3d image display device and method

ABSTRACT

A 3D image display device and method are provided. The 3D image display device divides a first 3D image into multiple depth layers, determines irregular pixels corresponding to the divided depth layers, generates second 3D images corresponding to the depth layers, respectively, using the corresponding irregular pixels, and synthesizes the generated images, thereby providing a final high-resolution 3D image.

This application claims the benefit under 35 U.S.C. §§111(a) and 363,and is a continuation of an International Application No.PCT/KR2015/004822 filed on May 14, 2015, which claims priority under 35U.S. C. §§119(e), 120 and 365(c) to Chinese Patent Application No.201410338749.7, filed on Jul. 16, 2014, in the State IntellectualProperty Office of the People's Republic of China, and to Korean PatentApplication No. 10-2015-0061078, filed on Apr. 30, 2015, in the KoreanIntellectual Property Office, the entire disclosure of which isincorporated herein by reference for all purposes.

BACKGROUND

1. Field

The following description relate to a method and apparatus fordisplaying a three-dimensional (3D) image using an irregular pixel.

2. Description of Related Art

A naked eye three-dimensional (3D) display apparatus includes a displaypanel to display a high-resolution two-dimensional (2D) interlaced imageand a light direction modulation element, such as, for example, amicro-lens array (MLA) to refract the interlaced image in a differentdirection, thereby providing a 3D image to be viewed by naked eyes,without a need to wear glasses. In the interlaced image includingmulti-view image information, adjacent pixels may display imageinformation at different angles. Each image may need to be separatedthrough a refraction of a lens to provide a clear 3D image. However, acrosstalk effect may occur in the adjacent pixels. Due to the crosstalkeffect, 3D images overlap each other, which may lead to a degradation inresolution. When the multi-view image is separated through therefraction of the lens, a ray of light may also be diffused in a processof propagating the ray of light radiated from a pixel. In thissituation, the adjacent pixels may be interfered with in response to achange in a beam area, which may also lead to a degradation inresolution.

Accordingly, in the naked eye 3D display system, a depth of field (DOF)may be restricted due to a characteristic that the ray of light isphysically propagated, and the resolution may be changed based on adepth layer.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In one general aspect, there is provided an apparatus for displaying athree-dimensional (3D) image, the apparatus including a depth layerdivider configured to divide a first 3D image into depth layers, a pixeldeterminer configured to determine irregular pixels respectivelycorresponding to the depth layers, a 3D image generator configured togenerate second 3D images respectively corresponding to the depth layersusing the determined irregular pixels, and a 3D image compositorconfigured to composite the second 3D images.

The depth layer divider may be configured to divide the first 3D imageinto the depth layers using a depth peeling algorithm.

The pixel determiner may be configured to set depth planes based on anoptical characteristic of a microlens array, and to determine theirregular pixels corresponding to the depth layers according to a depthplane to which a depth layer of the depth layers belongs.

The apparatus may include a contour detail feature extractor configuredto extract a contour detail feature from the first 3D image and toanalyze a frequency direction and a frequency magnitude of the contourdetail feature, wherein the pixel determiner may be configured todetermine the irregular pixels corresponding to the depth layers basedon any one or any combination of a frequency direction of the contourdetail feature corresponding to each of the depth layers, a frequencymagnitude of the contour detail feature corresponding to the each of thedepth layers, and the depth planes to which the depth layersrespectively belong.

The 3D image generator may be configured to render multi-view imagesusing the determined irregular pixels respectively corresponding to thedepth layers based on multi-view image information, to rearrange pixelswith respect to the rendered multi-view images, and to generate thesecond 3D images respectively corresponding to the depth layers.

The multi-view image information may include any one or any combinationof viewpoint position information and gaze field angle information.

The 3D image compositor may be configured to determine a back-and-forthlocation relationship in a depth direction for different portions of thesecond 3D images, and to composite the second 3D images in an order froma deepest layer based on the determined back-and-forth locationrelationship.

The pixel determiner may be configured to select an irregular pixel foreach of the depth layers from irregular pixels set in advance.

The irregular pixels may each be a pixel block including adjacentregular pixels or sub-pixels.

The irregular pixels may be different from the regular pixels in shapeand size.

In another general aspect, there is provided a method of displaying athree-dimensional (3D) image, the method including dividing a first 3Dimage into depth layers, determining irregular pixels respectivelycorresponding to the depth layers, generating second 3D imagesrespectively corresponding to the depth layers using the determinedirregular pixels, and compositing the generated second 3D images.

The dividing may include dividing the first 3D image into the depthlayers using a depth peeling algorithm.

The determining may include setting depth planes based on an opticalcharacteristic of a microlens array, and determining the irregularpixels respectively corresponding to the depth layers according to adepth plane to which a depth layer of the depth layers belongs.

The method may include extracting a contour detail feature from thefirst 3D image and determining a frequency direction and a frequencymagnitude of the contour detail feature, wherein the determining of theirregular pixels may include determining the irregular pixelscorresponding to the depth layers based on any one or any combination ofa frequency direction of the contour detail feature corresponding toeach of the depth layers, a frequency magnitude of the contour detailfeature corresponding to the each of the depth layers, and the depthplanes to which the depth layers respectively belong.

The generating may include rendering a plurality of multi-view imagesusing the determined irregular pixels respectively corresponding to thedepth layers based on multi-view image information, rearranging pixelswith respect to the rendered multi-view images, and generating thesecond 3D images respectively corresponding to the depth layers.

The multi-view image information may include any one or any combinationof viewpoint position information and gaze field angle information.

The compositing may include determining a back-and-forth locationrelationship in a depth direction for different portions of the second3D images, and compositing the second 3D images in an order from adeepest layer based on the determined back-and-forth locationrelationship to acquire a final 3D image.

The determining may include selecting an irregular pixel for each of thedepth layers from irregular pixels set in advance.

The irregular pixels may be each a pixel block comprises adjacentregular pixels or sub-pixels.

The irregular pixels may be different from the regular pixels in shapeand size.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates an example of a modulation transfer function (MTF) toestimate a display resolution.

FIG. 2 is a diagram illustrating an example of a three-dimensional (3D)image display apparatus.

FIG. 3 illustrates an example of a result of depth peeling.

FIG. 4 illustrates an example of irregular pixels.

FIGS. 5A and 5B illustrate examples of comparing a 3D image generated bya 3D image display apparatus to a general 3D image.

FIG. 6 illustrates a principle of reducing crosstalk by displaying a 3Dimage based on irregular pixels.

FIG. 7 is a diagram illustrating an example of a method of displaying a3D image.

FIG. 8 is a diagram illustrating an example of MTF value obtainedthrough a display test conducted based on different irregular pixelsusing a 3D image display apparatus.

FIG. 9 illustrates an example of an result of an analysis performed bycomparing an MTF value obtained through a display test using irregularpixels to an MTF value obtained through a display test using regularpixels.

Throughout the drawings and the detailed description, unless otherwisedescribed, the same drawing reference numerals will be understood torefer to the same elements, features, and structures. The relative sizeand depiction of these elements may be exaggerated for clarity,illustration, and convenience.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or apparatuses described herein. However, various changes,modifications, and equivalents of the methods, apparatuses, and/orapparatuses described herein will be apparent after an understanding ofthe disclosure of this application. For example, the sequences ofoperations described herein are merely examples, and are not limited tothose set forth herein, but may be changed as will be apparent after anunderstanding of the disclosure of this application, with the exceptionof operations necessarily occurring in a certain order. Also,descriptions of features that are known in the art may be omitted forincreased clarity and conciseness.

The features described herein may be embodied in different forms, andare not to be construed as being limited to the examples describedherein. Rather, the examples described herein have been provided merelyto illustrate some of the many possible ways of implementing themethods, apparatuses, and/or apparatuses described herein that will beapparent after an understanding of the disclosure of this application.

FIG. 1 illustrates an example of a modulation transfer function (MTF) toestimate a display resolution. In an example, a performance of anaked-eye three-dimensional (3D) display apparatus is measured based ona display resolution and a depth of field (DOF) of a 3D image. Forexample, the display resolution of the 3D image is quantized using anMTF. The MTF may measure a resolution based on a number of lines in arange of about 1 millimeter (mm), and may be defined as, “MTFvalue=(Maximal value of illuminance−Minimal value ofilluminance)/(Maximal value of illuminance+Minimal value ofilluminance)”. In an example, the MTF value is a value between 0 and 1.For example, the greater the MTF value, the higher the resolution of the3D image that is currently displayed, and the lesser the MTF value, thelower the resolution of the 3D image.

A graph of FIG. 1 represents an MTF of a 3D image displayed based on asinusoidal grating. In the graph, an X-axial direction represents adepth value of the 3D image, a Y-axial direction represents a frequency,and a Z-axial direction represents an MTF value. In an example, thesinusoidal grating includes different frequencies and different depthplanes. In an example of FIG. 1, a resolution is determined based on acentral depth plane (CDP). The resolution is maximized on the CDP, andthe resolution decreases according to an increase in a distance from theCDP, which may lead to a crosstalk error and a blur effect. In otherwords, in the naked-eye 3D display apparatus, the DOF of the displayed3D image may be restricted due to a physical light propagationcharacteristic.

Accordingly, an issue of the physical light propagation characteristicsuch as crosstalk may need to be solved to provide a high-resolution 3Dimage in a naked-eye 3D display apparatus.

FIG. 2 is a diagram illustrating an example of a 3D image displayapparatus 100. Referring to FIG. 2, the 3D image display apparatus 100includes a depth layer divider 110, a pixel determiner 120, a 3D imagegenerator 130, and a 3D image compositor 140.

In an example, the depth layer divider 110 divides a first 3D imagecorresponding to a desired image to be displayed into a plurality ofdepth layers. For example, the depth layer divider 110 divides the first3D image into a number of depth layers using a depth peeling algorithm.In an example, the number of depth layers are predetermined.

In an example, the depth layer divider 110 converts a pixel included inthe first 3D image into a fragment. For example, a pixel including ahorizontal coordinate value x and a vertical coordinate value y isconverted into a pixel including the horizontal coordinate value x, thevertical coordinate value y, and a depth-directional coordinate value z,i.e., the fragment. In an example, the depth-directional coordinatevalue z indicates a depth may be a depth value of the pixelcorresponding to the fragment. Hereinafter, the depth value of the pixelcorresponding to the fragment may also be referred to as the depth valueof the fragment. The depth layer divider 110 may arrange fragments in adepth direction based on a depth value of each of the fragments,generate a plurality of depth layers based on a result of deptharrangement and a preset number of depth layers, and output thegenerated depth layers. For example, when a 3D image having a maximaldepth value is 4 and a minimal depth value is −4 is divided into fourdepth layers, a fragment having a depth value between 2 and 4 may bedivided as a first depth layer, a fragment having a depth value between0 and 2 may be divided as a second depth layer, a fragment having adepth value between −2 and 0 may be divided as a third depth layer, anda fragment having a depth value between −4 and −2 may be divided as afourth depth layer. Through this, a plurality of depth layers may beobtained to be output. The number of depth layers and a DOF of each ofthe depth layers may be varied without departing from the spirit andscope of the illustrative examples described. In the foregoing example,a fragment having a depth value between 3 and 4 may also be divided asthe first depth layer, a fragment having a depth value between 0 and 3may also be divided as the second depth layer, a fragment having a depthvalue between −3 and 0 may also be divided as the third depth layer, anda fragment having a depth value between −3 and −4 may also be divided asthe fourth depth layer.

FIG. 3 illustrates an example of a result of depth peeling. Asillustrated in FIG. 3, a depth peeling may be performed on a 3D image ofFIG. 3A using the depth layer divider 110 to form a plurality of depthlayers, as shown in of FIG. 3B. Through this, each of the depth layersmay represent a portion corresponding to the 3D image.

The depth layer divider 110 is configured to divide a 3D image to bedisplayed into a plurality of depth layers using a depth peelingalgorithm in the present example, other algorithms may be used withoutdeparting from the spirit and scope of the illustrative examplesdescribed. For example, a salience mapping method may be applied to thedepth layer division.

Referring back to FIG. 2, the pixel determiner 120 determines irregularpixels respectively corresponding to depth layers obtained by the depthlayer divider 110. A plurality of depth planes, which is a plurality ofDOFs, may be set based on an optical characteristic of a light directionmodulation element of the 3D image display apparatus 100, i.e., amicrolens array included in a 3D image display apparatus. For example, aCDP of FIG. 1 may be set to be a first depth layer, a plane on which adepth other than a depth corresponding to the CDP in a display of the 3Dimage display apparatus 100 is located may be set to be a second depthplane, and a plane on which a depth other than that of the display DOFof the 3D image display apparatus 100 is located may be set to be athird depth plane. In an example, the pixel determiner 120 determinesdepth planes to which the depth layers belong based on the DOFs of thedepth layers, and determine irregular pixels respectively correspondingto the depth layers based on the depth planes.

In an example, a predetermined depth layer L included in the depthlayers will be described. When the depth layer L belongs to the firstdepth plane, i.e., the CDP, an image displayed on a display plane mayneed to have a maximal resolution such that a 3D image portioncorresponding to the depth layer is displayed using general regularpixels. When the depth layer L belongs to the second depth plane, thatis, the plane on which the depth other than the depth corresponding tothe CDP in the display DOF of the 3D image display apparatus 100 islocated, or belongs to the third depth plane, i.e., the plane on whichthe depth other than that of the display DOF of the 3D image displayapparatus 100, the pixel determiner 120 determined an irregular pixelcorresponding to the depth layer L.

FIG. 4 illustrates an example of irregular pixels. The irregular pixelsmay be a pixel block including a plurality of adjacent regular pixels orsub-pixels and different from the regular pixels in shape and size. Forexample, a plurality of irregular pixels may be set in advance, and thepixel determiner 120 may select at least one irregular pixel for a depthlayer L from the preset irregular pixels. In an example, the pixeldeterminer 120 may compare resolutions obtained when a 3D image portioncorresponding to the depth layer L is displayed using various types ofirregular pixels and determine which irregular pixel is to be used. Thepixel determiner 120 may selectively determine the irregular pixel to beused by comparing the resolutions obtained when the 3D image portioncorresponding to the depth layer L is displayed using a combination ofat least two types of irregular pixels among the preset irregularpixels. In an example, the pixel determiner 120 combines adjacentregular pixels or sub-pixels to form a plurality of candidate irregularpixels in different shape and size. Using the candidate irregularpixels, in an example, the pixel determiner 120 determines a resolutionof a 3D image portion corresponding to the depth layer L, and determinesa candidate irregular pixel corresponding to a maximal resolution to bea final irregular pixel corresponding to the depth layer L.

In the foregoing, determining irregular pixels using a depth applyingmethod, i.e., determining irregular pixels respectively corresponding todepth layers based on a depth plane to which each of the depth layerbelongs is described, embodiments are not limited thereto and othermethods may be used without departing from the spirit and scope of theillustrative examples described.

In an example, the irregular pixels of the depth layers are determinedusing a frequency applying method. The irregular pixels of the depthlayers are determined based on a frequency feature, for example, adirection and a magnitude of a frequency of a 3D image to be displayed.The 3D image display apparatus 100 may include a contour detail featureextractor (not shown) to extract a contour detail feature from a first3D image and determine a frequency magnitude and a direction of thecontour detail feature, i.e., a frequency direction. In an example, thepixel determiner 120 determines the irregular pixels corresponding tothe depth layers based on any one or any combination of the direction ofthe contour detail feature corresponding to each of the depth layers,i.e., a contour detail feature of a pixel corresponding to a depthlayer, and the frequency magnitude of the contour detail featurecorresponding to each of the depth layers such that different widths indifferent frequency directions are implemented in a result thereof, forexample, a pixel width being inversely proportional to the frequencymagnitude.

A contour detail feature included in a 3D image may be extracted usingvarious methods, such as, for example, multi-scale Gabor filteringwithout departing from the spirit and scope of the illustrative examplesdescribed. Thus, related descriptions will be omitted.

In an example, the pixel determiner 120 determines the irregular pixelscorresponding to the depth layers using both depth applying method andfrequency applying method. The pixel determiner 120 determines theirregular pixels respectively corresponding to the depth layers based onat least one of the direction of the contour detail featurecorresponding to each of the depth layers, the frequency magnitudecorresponding to each of the depth layers, and the depth plane to whicheach of the depth layer belongs.

Although the divided depth layers and the set depth planes differ inDOF, in an example, the depth layer divider 110 may use depth layers anddepth planes having the same DOF in a process of depth layer division.

Referring back to FIG. 1, the 3D image generator 130 generates second 3Dimages respectively corresponding to the depth layers using theirregular pixels respectively corresponding to the depth layersdetermined by the pixel determiner 120.

In an example, the 3D image generator 130 renders a plurality ofmulti-view images using the irregular pixels corresponding to thedetermined depth layers based on multi-view image information of the 3Dimage display apparatus 100. In an example, the 3D image generator 130performs pixel rearrangement on the rendered multi-view images, therebygenerating the second 3D images corresponding to the depth layers. In anexample, the multi-view image information may include at least one ofviewpoint position information and field angle information and may be avalue corresponding to a hardware performance parameter setting of the3D image display apparatus 100. Each of the multi-view images maycorrespond to a single viewpoint and a single field angle position.

Rendering the multi-view images and performing the pixel rearrangementon the multi-view images may be performed by any of the various methods,and thus, related descriptions will be omitted. without departing fromthe spirit and scope of the illustrative examples

The 3D image compositor 140 may composite the plurality of second 3Dimages respectively corresponding to the plurality of depth layersgenerated by the 3D image generator 130, thereby acquiring a final 3Dimage.

In an example, the 3D image compositor 140 determines a back-and-forthlocation relationship in a depth direction with respect to differentportions of the plurality of second 3D images respectively correspondingto the plurality of depth layers generated by the 3D image generator130. The 3D image compositor composites the plurality of second 3Dimages in an order from a deepest layer, for example, covering from theback to front, based on the determined back-and-forth locationrelationship, thereby acquiring the final 3D image. When compositing thesecond 3D images, the compositing may be performed in an order from a 3Dimage of the deepest depth layer. In an example an image of eachlocation included in the final 3D image is determined based on acorresponding location of the second 3D image having a minimal depthamong the plurality of second 3D images used to composite the final 3Dimage.

FIG. 5A and 5B illustrate examples of comparing a 3D image generated bythe 3D image display apparatus 100 to a general 3D image. FIG. 5A)illustrates a screenshot of a 3D image having a depth and displayedusing regular pixels based on general technology, and FIG. 5Billustrates the same 3D image displayed using the 3D image displayapparatus 100. As illustrated in FIG. 5, in comparison to a result ofdisplaying the 3D image using the regular pixels based on the generaltechnology, the 3D image generated by the 3D image display apparatus 100may be displayed with increased resolution and a display DOF.

FIG. 6 illustrates an example of reducing crosstalk by displaying a 3Dimage based on irregular pixels according to an example embodiment. Aleft portion of FIG. 6 illustrates a screen on which a 3D image isdisplayed using regular pixels. A general apparatus of displaying a 3Dimage using regular pixels may be restricted on a hardware design. Thus,a user's eye may observe luminous intensity of two adjacent pixels. Inthis example, the two pixel may represent different viewpoint images andthus, a crosstalk effect may occur. A right portion of FIG. 6illustrates an example of displaying a 3D image using irregular pixels.The irregular pixels used in the right portion of FIG. 6 are greater insize than the regular pixels. In this example, the user may observeluminous intensity of one pixel with the eyes at the same location asthe left portion of FIG. 6 and thus, the crosstalk effect may not occur.Accordingly, the crosstalk effect occurring in a process of displayingthe 3D image may be alleviated.

FIG. 7 is a diagram illustrating an example of a method of displaying a3D image. The operations in FIG. 7 may be performed in the sequence andmanner as shown, although the order of some operations may be changed orsome of the operations omitted without departing from the spirit andscope of the illustrative examples described. Many of the operationsshown in FIG. 7 may be performed in parallel or concurrently. Inaddition to the description of FIG. 7 below, the above descriptions ofFIGS. 1-6, are also applicable to FIG. 7, and are incorporated herein byreference. Thus, the above description may not be repeated here.

In 710, the depth layer divider 110 included in the 3D image displayapparatus 100 divides a first 3D image into a plurality of depth layers.In an example, depth layer divider 110 divides a 3D image into aplurality of depth layers using a depth peeling algorithm. A pixelincluded in the first 3D image may be converted into a fragment, i.e., apixel including a horizontal coordinate value x and a verticalcoordinate value y may be converted into a pixel including thehorizontal coordinate value x, the vertical coordinate value y, and adepth-directional coordinate value z corresponding to a depth value ofthe pixel, i.e., the fragment. Thus, fragments may be arranged in adepth direction based on a depth value of each of the fragment. Based ona result of depth arrangement and the preset number of depth layers, aplurality of depth layers may be generated and output.

In 730, the pixel determiner 120 included in the 3D image displayapparatus 100 determines irregular pixels corresponding to the depthlayers into which the first 3D image is divided by the depth layerdivider 110. The irregular pixels may each be a pixel block including aplurality of adjacent regular pixels or sub-pixels. In this example, theirregular pixels may be different from the regular pixels in shape andsize. An operation of determining irregular pixels may select at leastone irregular pixel for each of the depth layers from a plurality ofirregular pixels set in advance.

In an example, a plurality of depth planes may be set based on anoptical characteristic of a microlens array, which is a light directionmodulation element of the 3D image display apparatus 100. An operationof determining the irregular pixels may determine irregular pixelsrespectively corresponding to depth layers based on a depth plane towhich each of the depth layers belongs. The descriptions of FIG. 2 arealso applicable here, and are incorporated herein by reference. Thus,the above description with respect to a method of setting a depth planeto which a depth layer belongs and determining irregular pixels based onthe depth plane may not be repeated here.

In 730, the irregular pixels of the depth layers may be determined basedon a frequency characteristic, such as, for example, a direction and amagnitude of a frequency of the first 3D image. Operation 730 mayinclude an operation of extracting a contour detail feature from thefirst 3D image and determining a frequency direction and a frequencymagnitude of the contour detail feature. In this example, operation 730determines the irregular pixels respectively corresponding to the depthlayers based on at least one of the frequency direction of the contourdetail feature corresponding to each of the depth layers, the frequencymagnitude of the contour detail feature corresponding to each of thedepth layers, and the depth planes to which the depth layersrespectively belong.

In 750, the 3D image generator 130 included in the 3D image displayapparatus 100 generates second 3D images respectively corresponding tothe depth layers using the irregular pixels respectively correspondingto the depth layers determined in operation 730. In an example, aplurality of multi-view images are rendered using the irregular pixelsrespectively corresponding to the depth layers determined in operation730 based on multi-view image information of the 3D image displayapparatus 100, and pixel arrangement may be performed on the renderedmulti-view images. Through this, 3D images corresponding to the depthlayers may be generated. In an example, the multi-view image informationmay include at least one of viewpoint location information and gazefield angle information.

In 770, the 3D image compositor 140 of the 3D image display apparatus100 composites the second 3D images to obtains a final 3D image. Forexample, a back-and-forth location relationship in a depth direction maybe determined with respect to different portions of the second 3D imagesgenerated in operation 750. Based on the determined back-and-forthlocation relationship, the second 3D images may be composited in anorder from a deepest layer. Through this, the final 3D image may beobtained.

FIG. 8 is a diagram illustrating an example of an MTF value obtainedthrough a display test conducted based on different irregular pixelsusing a 3D image display apparatus. In FIG. 8, a graph with a symbol “

” represents an MTF value obtained through a display using regularpixels and graphs with other symbols represent MTF values obtainedthrough a display using irregular pixels.

As illustrated in FIG. 8, irregular pixels having different shapes maycorrespond to different MTF values. However, it is indicated that mostof the irregular pixels enhances a resolution of a portion of depthplanes other than the CDP, that is, the depth plane corresponding to themaximal MTF value in the example of FIG. 8.

FIG. 9 illustrates an example of a comparison of an MTF value obtainedthrough a display test using irregular pixels to an MTF value obtainedthrough a display test using regular pixels. In FIG. 9, a test frequencymay be 0.097 period/mm. As illustrated in FIG. 9, using irregularpixels, a resolution may increase according to an increase in a distancefrom a CDP while a depth is 10 mm.

Accordingly, the 3D image display apparatus and method may solve acrosstalk error, enhance a resolution of a 3D display, increase a DOF ofa 3D image to be displayed, and increase a speed of processing the 3Dimage. Also, manufacturing costs of the 3D image display apparatus maybe reduced.

The 3D image display apparatus 100, pixel determiner 120, 3D imagegenerator 130, 3D image compositor 140, and other units and apparatusesdescribed in the FIGS., for example FIG. 1, which perform the operationsdescribed in this application are implemented by hardware componentsconfigured to perform the operations described in this application.Examples of hardware components that may be used to perform theoperations described in this application where appropriate includecontrollers, sensors, generators, drivers, memories, comparators,arithmetic logic units, adders, subtractors, multipliers, dividers,integrators, and any other electronic components configured to performthe operations described in this application. In other examples, one ormore of the hardware components that perform the operations described inthis application are implemented by computing hardware, for example, byone or more processors or computers. A processor or computer may beimplemented by one or more processing elements, such as an array oflogic gates, a controller and an arithmetic logic unit, a digital signalprocessor, a microcomputer, a programmable logic controller, afield-programmable gate array, a programmable logic array, amicroprocessor, or any other device or combination of devices that isconfigured to respond to and execute instructions in a defined manner toachieve a desired result. In one example, a processor or computerincludes, or is connected to, one or more memories storing instructionsor software that are executed by the processor or computer. Hardwarecomponents implemented by a processor or computer may executeinstructions or software, such as an operating system (OS) and one ormore software applications that run on the OS, to perform the operationsdescribed in this application. The hardware components may also access,manipulate, process, create, and store data in response to execution ofthe instructions or software. For simplicity, the singular term“processor” or “computer” may be used in the description of the examplesdescribed in this application, but in other examples multiple processorsor computers may be used, or a processor or computer may includemultiple processing elements, or multiple types of processing elements,or both. For example, a single hardware component or two or morehardware components may be implemented by a single processor, or two ormore processors, or a processor and a controller. One or more hardwarecomponents may be implemented by one or more processors, or a processorand a controller, and one or more other hardware components may beimplemented by one or more other processors, or another processor andanother controller. One or more processors, or a processor and acontroller, may implement a single hardware component, or two or morehardware components. A hardware component may have any one or more ofdifferent processing configurations, examples of which include a singleprocessor, independent processors, parallel processors,single-instruction single-data (SISD) multiprocessing,single-instruction multiple-data (SIMD) multiprocessing,multiple-instruction single-data (MISD) multiprocessing, andmultiple-instruction multiple-data (M I M D) multiprocessing.

The methods illustrated in FIG. 7 that perform the operations describedin this application are performed by computing hardware, for example, byone or more processors or computers, implemented as described aboveexecuting instructions or software to perform the operations describedin this application that are performed by the methods. For example, asingle operation or two or more operations may be performed by a singleprocessor, or two or more processors, or a processor and a controller.One or more operations may be performed by one or more processors, or aprocessor and a controller, and one or more other operations may beperformed by one or more other processors, or another processor andanother controller. One or more processors, or a processor and acontroller, may perform a single operation, or two or more operations.

The instructions or software to control a processor or computer toimplement the hardware components and perform the methods as describedabove, and any associated data, data files, and data structures, arerecorded, stored, or fixed in or on one or more non-transitorycomputer-readable storage media. Examples of a non-transitorycomputer-readable storage medium include read-only memory (ROM),programmable read only memory (PROM), electrically erasable programmableread-only memory (EEPROM), random-access memory (RAM), dynamic randomaccess memory (DRAM), static random access memory (SRAM), flash memory,non-volatile memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs,DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs,BD-REs, blue-ray or optical disk storage, hard disk drive (HDD), solidstate drive (SSD), flash memory, a card type memory such as multimediacard micro or a card (for example, secure digital (SD) or extremedigital (XD)), magnetic tapes, floppy disks, magneto-optical datastorage devices, optical data storage devices, hard disks, solid-statedisks, and any other device that is configured to store the instructionsor software and any associated data, data files, and data structures ina non-transitory manner and providing the instructions or software andany associated data, data files, and data structures to a processor orcomputer so that the processor or computer can execute the instructions.In one example, the instructions or software and any associated data,data files, and data structures are distributed over network-coupledcomputer systems so that the instructions and software and anyassociated data, data files, and data structures are stored, accessed,and executed in a distributed fashion by the processor or computer.

While this disclosure includes specific examples, it will be apparent toone of ordinary skill in the art that various changes in form anddetails may be made in these examples without departing from the spiritand scope of the claims and their equivalents. The examples describedherein are to be considered in a descriptive sense only, and not forpurposes of limitation. Descriptions of features or aspects in eachexample are to be considered as being applicable to similar features oraspects in other examples. Suitable results may be achieved if thedescribed techniques are performed in a different order, and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner, and/or replaced or supplemented by othercomponents or their equivalents. Therefore, the scope of the disclosureis defined not by the detailed description, but by the claims and theirequivalents, and all variations within the scope of the claims and theirequivalents are to be construed as being included in the disclosure.

1. An apparatus for displaying a three-dimensional (3D) image, theapparatus comprising: a depth layer divider configured to divide a first3D image into depth layers; a pixel determiner configured to determineirregular pixels respectively corresponding to the depth layers; a 3Dimage generator configured to generate second 3D images respectivelycorresponding to the depth layers using the determined irregular pixels;and a 3D image compositor configured to composite the second 3D images.2. The apparatus of claim 1, wherein the depth layer divider is furtherconfigured to divide the first 3D image into the depth layers using adepth peeling algorithm.
 3. The apparatus of claim 1, wherein the pixeldeterminer is further configured to set depth planes based on an opticalcharacteristic of a microlens array, and to determine the irregularpixels corresponding to the depth layers according to a depth plane towhich a depth layer of the depth layers belongs.
 4. The apparatus ofclaim 3, further comprising: a contour detail feature extractorconfigured to extract a contour detail feature from the first 3D imageand to analyze a frequency direction and a frequency magnitude of thecontour detail feature, wherein the pixel determiner is furtherconfigured to determine the irregular pixels corresponding to the depthlayers based on any one or any combination of a frequency direction ofthe contour detail feature corresponding to each of the depth layers, afrequency magnitude of the contour detail feature corresponding to theeach of the depth layers, and the depth planes to which the depth layersrespectively belong.
 5. The apparatus of claim 1, wherein the 3D imagegenerator is further configured to render multi-view images using thedetermined irregular pixels respectively corresponding to the depthlayers based on multi-view image information, to rearrange pixels withrespect to the rendered multi-view images, and to generate the second 3Dimages respectively corresponding to the depth layers.
 6. The apparatusof claim 5, wherein the multi-view image information comprises any oneor any combination of viewpoint position information and gaze fieldangle information.
 7. The apparatus of claim 1, wherein the 3D imagecompositor is further configured to determine a back-and-forth locationrelationship in a depth direction for different portions of the second3D images, and to composite the second 3D images in an order from adeepest layer based on the determined back-and-forth locationrelationship.
 8. The apparatus of claim 1, wherein the pixel determineris further configured to select an irregular pixel for each of the depthlayers from irregular pixels set in advance.
 9. The apparatus of any oneof claim 1, wherein the irregular pixels are each a pixel blockcomprising adjacent regular pixels or sub-pixels.
 10. The apparatus ofclaim 9, wherein the irregular pixels are different from the regularpixels in shape and size.
 11. A method of displaying a three-dimensional(3D) image, the method comprising: dividing a first 3D image into depthlayers; determining irregular pixels respectively corresponding to thedepth layers; generating second 3D images respectively corresponding tothe depth layers using the determined irregular pixels; and compositingthe generated second 3D images.
 12. The method of claim 11, wherein thedividing comprises dividing the first 3D image into the depth layersusing a depth peeling algorithm.
 13. The method of claim 11, wherein thedetermining comprises setting depth planes based on an opticalcharacteristic of a microlens array, and determining the irregularpixels respectively corresponding to the depth layers according to adepth plane to which a depth layer of the depth layers belongs.
 14. Themethod of claim 13, further comprising: extracting a contour detailfeature from the first 3D image and determining a frequency directionand a frequency magnitude of the contour detail feature, wherein thedetermining of the irregular pixels comprises determining the irregularpixels corresponding to the depth layers based on any one or anycombination of a frequency direction of the contour detail featurecorresponding to each of the depth layers, a frequency magnitude of thecontour detail feature corresponding to the each of the depth layers,and the depth planes to which the depth layers respectively belong. 15.The method of claim 11, wherein the generating comprises rendering aplurality of multi-view images using the determined irregular pixelsrespectively corresponding to the depth layers based on multi-view imageinformation, rearranging pixels with respect to the rendered multi-viewimages, and generating the second 3D images respectively correspondingto the depth layers.
 16. The method of claim 15, wherein the multi-viewimage information comprises any one or any combination of viewpointposition information and gaze field angle information.
 17. The method ofclaim 11, wherein the composition comprises determining a back-and-forthlocation relationship in a depth direction for different portions of thesecond 3D images, and compositing the second 3D images in an order froma deepest layer based on the determined back-and-forth locationrelationship to acquire a final 3D image.
 18. The method of claim 11,wherein the determining comprises selecting an irregular pixel for eachof the depth layers from irregular pixels set in advance.
 19. The methodof any one of claims 11, wherein the irregular pixels are each a pixelblock comprises adjacent regular pixels or sub-pixels.
 20. The method ofclaim 19, the irregular pixels are different from the regular pixels inshape and size.